Strategies for reducing the complexity of a RNN based speech recognizer

Recurrent Neural Networks (RNN) provide a solution for low cost Speech Recognition Systems (SRS) in mass products or in products with energetic constraints if their inherent parallelism could be exploited in a hardware realization. Actually, the computational complexity of SRS based on Fully Recurrent Neural Networks (FRNN), e.g. the large number of connections, prevents a hardware realization. Here we introduce Locally Recurrent Neural Networks (LRNN) in order to keep the properties of RNN on the one hand and to reduce the connectivity density of the network on the other hand. By simulation experiments it is shown that the recognition capability of LRNN is equivelant to that of FRNN and superior to other proposed network archi-tectures. Furthermore, it is shown that with an appropriate representation of the network paramaters and a retraining of the network 5 Bit quantization of the weights and activities is possipble without signiicant loss in recognition performance.